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ABSTRACT 

Head  Policy,  Guidance  and  Analysis  (HPGA)  seeks  to  develop  a  strategic  planning  framework  to 
better  support  Defence  decision-making.  This  paper  was  written  in  response  to  HPGA's  request 
for  advice  on  the  potential  role  of  Operations  Research  (OR)  and  analysis  in  such  a  framework.  A 
broad  interpretation  of  OR  is  proposed  that  includes  qualitative  analysis  of  problems  that  are  not 
amenable  to  the  quantitative  methods  with  which  OR  is  more  traditionally  associated.  Cultural 
and  ownership  issues  that  must  be  addressed  if  OR  is  to  be  successfully  incorporated  into 
Defence's  decision-making  processes  are  discussed.  It  is  argued  that  OR  and  analysis  should  be 
conducted  with  a  decision  perspective,  and  that  analysis  should  be  embedded  in  all  aspects  of  a 
strategic  planning  framework.  Working  definitions  of  terms  and  concepts  are  presented  to 
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The  Role  of  Operations  Research  and  Analysis 
in  Concept-led  Long-Range  Planning 


Executive  Summary 

Head  Policy,  Guidance  and  Analysis  (HPGA)  and  Director-General  Military  Strategy 
(DGMS)  seek  to  build  a  strategic  planning  framework  that  will  provide  improved  decision 
support  for  senior  Defence  committees  and  decision-makers.  This  paper  has  been  written 
in  response  to  their  request  for  advice  on  the  potential  role  of  Operations  Research  (OR) 
and  analysis  in  this  framework, 

OR  is  "the  application  of  the  methods  of  science  to  complex  problems  involving  human 
and  technical  factors,  with  the  aim  of  improving  decisions"1.  A  broad  interpretation  of  OR 
is  proposed  to  include  qualitative  analysis  of  problems  that  are  not  amenable  to  the 
quantitative  methods  with  which  OR  is  more  traditionally  associated.  With  this 
interpretation,  OR  is  considered  as  a  rigorous  and  structured  approach  to  providing 
decision  support,  where  the  nature  of  the  analysis,  the  tools  used  and  the  characteristics  of 
the  answers  are  dependen  t  on  the  type  of  problem  to  be  addressed.  For  a  concept-led  long- 
range  planning  framework  many  of  the  problems  to  be  addressed  are  not  likely  to  be 
amenable  to  quantitative  analysis,  but  can  be  addressed  rigorously  with  qualitative 
analysis.  In  contrast,  resource  decisions  to  be  made  by  senior  committees  are  more  likely 
to  require  quantitative  analysis.  In  either  case,  the  broader  interpretation  of  OR  can 
provide  value  by  bringing  rigour  to  the  development  of  decision  support  information. 

Hie  incorporation  of  OR  and  analysis  into  the  long-range  planning  process  raises 
ownership  and  cultural  issues.  These  issues  are  various,  depending  upon  how  OR  is 
incorporated  into  the  organisation.  Of  particular  significance  are:  the  degree  to  which 
objective  and  impartial  analysis  is  desired  by  the  organisation,  the  arrangements  for 
maintaining  an  analytical  capability  and  the  degree  to  which  the  organisational  culture 
allows  open  and  robust  debate. 

The  argument  is  made  that  OR  can  provide  the  greatest  contribution  to  the  long-range 
planning  framework  if  a  decision  perspective  is  adopted.  That  is,  analysis  is  performed 
with  explicit  links  to  actual  or  potential  decisions.  In  practice,  this  means  that  analysis  is 
focussed  on  providing  decision  support  information  of  direct  relevance  to  decision¬ 
makers,  rather  than  simply  for  purposes  of  discovery.  Embedding  analysis  in  all  parts  of 
the  long-range  planning  process  will  assist  in  achieving  that  aim. 


1  Agrell,  P.S.  “Operational  Research  at  the.  National  Defence  Research  Institute  of  Sweden”,  Omega,  Vol  13 
No.  2,  p  121-124  1985 
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1.  Introduction 


1.  Head  Policy,  Guidance  and  Analysis  (HPGA)  and  Director-General  Military  Strategy 
(DGMS)  seek  to  build  a  strategic  planning  framework  that  will  provide  improved  decision 
support  for  senior  Defence  committees  and  decision-makers.  Specifically,  the  focus  is  on 
producing  a  concept-led  long-range  planning  process.  HPG  A's  view  is  that  the  framework 
will  build  on  extant  activities  and  experience  as  far  as  possible.  It  may  be  necessary  to 
optimise  existing  elements  of  the  framework  and  their  interrelationships,  and  to 
implement  and  develop  elements  that  do  not  currently  exist. 

2.  The  ambitious  nature  of  this  undertaking,  together  with  the  fact  that  there  is  not  a 
"clean  slate"  to  work  with,  means  that  HPG  A/DGMS  need  to  drive  the  development  and 
implementation  of  the  framework  hard  and  fast  to  ensure  that  it  has  sufficient  momentum 
and  progress  to  survive  beyond  the  current  posting  cycle.  They  have  requested  DSTO  to 
provide  information  to  help  focus  PGA,  DSTO,  Services  and  private  sector  effort  in 
developing  and  implementing  the  framework. 

3.  This  information/ discussion  paper  is  provided  in  response  to  a  request  from 
HPGA/DGMS  for  advice  on  the  potential  role  of  operations  research  and  analysis  in  a 
long-range  planning  process. 

4.  Scope.  This  paper  presents  a  discussion  of  what  is  meant  by  the  term  "Operations 
Research",  how  it  could  contribute  to  long-range  planning  in  Defence,  some  organisational 
implications  of  operations  research  and  how  it  might  be  integrated  into  a  concept-led  long- 
range  planning  framework.  The  paper  also  presents  definitions  of  terms  and  concepts  that 
are  intended  to  provide  a  common  lexicon  for  those  involved  in  the  development  of  long- 
range  planning. 

5.  Aim.  The  aim  of  this  paper  is  to  provide  HPGA  and  DGMS  with  information  and 
discussion  that  will  inform  their  decisions  on  the  potential  role  of  operations  research  and 
analysis  in  the  development  and  implementation  of  a  framework  for  concept-led  long- 
range  planning  in  Defence. 


2.  Definitions 

6.  Operations  Research  (OR): " the  application  of  the  methods  of  science  to  complex  problems 
involving  human  and  technical  factors,  with  the  aim  of  improving  decisions"1. 

7.  OR  had  its  origins  in  analysing  military  operations  at  the  tactical  level  during  the 
Second  World  War.  It  has  since  been  applied  beyond  the  military  sphere  to  include  the 
private  and  civilian  public  sectors,  and  in  the  realm  of  strategic  decision-making. 
Distinctions  are  often  made  between  the  terms  Operations  Research,  Operations  Analysis, 
Operational  Analysis,  Operational  Research,  System  Analysis,  and  Management  Science, 
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but  these  are  somewhat  academic,  and  for  the  purposes  of  this  paper  they  are  taken  to  be 
interchangeable. 

8.  Scientific  Method:  "the  process  of  reasoning — using  methods  of  induction,  deduction  and 
intuition  informed  through  observation  and  experience — to  formulate,  test,  analyse  and  review 
hypotheses  (and/or  posited  explanations)  about  the  nature  of  the  world  in  order  to  generate 
understanding".2 

9.  Further  discussion  of  the  scientific  method  is  given  later  in  the  text. 

10.  Decision:  "a  determination  arrived  at  after  consideration  of  several  possibilities "3. 

11.  In  the  Defence  context,  the  term  "decision"  often  refers  to  specific  senior  committee 
considerations  such  as  those  that  occur  at  well-defined  points  dining  the  acquisition  of  a 
new  capability.  In  the  context  of  Defence  long-range  planning,  and  for  the  purposes  of  this 
paper,  the  term  will  be  taken  to  have  a  wider  meaning  to  include  potential  decisions  that 
are  not  yet  on  any  current  decision  schedule. 

12.  Defence  Long-range  Planning:  "a  process  that  investigates  possible  future  operating 
environments  and  produces  a  strategy  to  best  fit  the  Defence  organisation  to  those  environments, 
given  a  host  of  constraints,  including  financial  ones "4. 

13.  The  timeframe  encompassed  by  the  phrase  "long-range"  is  not  precisely  defined,  but  is 
related  to  how  long  it  takes  to  make  changes  in  the  system  and  the  length  of  time  those 
changes  will  have  effects  on  the  system.  It  will  be  different  for  each  sector  of  Defence, 
depending,  for  example,  on  how  long  it  takes  to  make  changes  to  organisational  structures 
or  to  bring  new  platforms  into  service,  or  the  expected  service  life  of  a  given  platform.  In 
the  Defence  context  this  might  typically  be  a  10-30  year  horizon.  Defence  long-range 
planning  will  usually  include  considerations  of  the  context  within  which  the  plan  will 
operate,  concepts  that  inform  possible  ways  for  the  organisation  to  operate  now  and  into 
the  future  and  the  capability  implications  of  the  context  and  concepts. 

14.  Hypothesis: " supposition  made  as  a  basis  for  reasoning,  without  assumption  of  its  truth,  or  as 
a  starting  point  for  further  investigation  from  known  facts" . 5 

15.  In  the  present  context,  "hypothesis"  includes  suppositions  that  are  appropriate  for 
discovery  and  exploration,  not  just  those  that  can  be  used  for  statistical  interpretation  of 
"hard"  (quantitative)  experiments.  It  also  includes  the  idea  of  "layered  hypotheses",  in  which 
subordinate  hypotheses  of  increasing  detail  or  focus  are  proposed  so  that  experiments  can 
be  designed  to  examine  several  aspects  of  a  problem. 

16.  Experiment:  "procedure  adopted  to  test  hypothesis  or  demonstrate  known  fact". 6 

17.  The  design  of  experiments  (the  nature  of  the  methodology  adopted  and  the 
experimental  tools  used)  should  be  guided  by  the  nature  of  the  hypothesis  or  questions  to 
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be  addressed.  Experiments  may  be  conducted  using  back-of-the-envelope  calculations, 
computer  models,  simulations,  seminar  games,  campaign  models,  wargames  or  any 
number  of  other  experimental  tools.  The  essential  point  is  that  the  hypothesis  should  drive 
experimental  design,  not  the  other  way  around. 

18.  Analysis: " separation  of  a  whole,  whether  a  material  substance  or  any  matter  of  thought,  into 
its  constituent  elements  as  a  method  of  studying  the  nature  of  a  thing  or  of  determining  its  essential 
features"  7 

19.  "Analysis"  in  this  context  is  any  activity  that  contributes  to  the  understanding  of  a 
system.  The  dominant  paradigm  in  Defence  appears  to  be  that  analysis  is  a  separate 
activity  that  occurs  after  an  experiment  is  conducted.  This  paper  takes  the  view  that 
analysis  should  be  embedded  throughout  the  concept-led  long-range  planning  process, 
and  that  one  of  its  major  functions  is  to  guide  experimentation. 

20.  Model:  " simplified  description  of  any  system  etc.  to  assist  calculations  and  predictions ",s 

21.  Models  are  frequently  misunderstood  to  embody  some  truth  or  reality  about  the 
system  they  describe.  They  should  be  regarded  as  a  representation  of  the  phenomenon 
being  studied  that  is  adequate  for  the  purpose  of  the  study.  While  some  models  may  be 
expressed  in  a  mathematical  form,  many  are  non-mathematical. 


3.  What  is  Operations  Research? 

22.  A  more  expansive  description  of  OR  is: 

"a  systematic  approach  to  helping  a  decision  maker  choose  a  course  of  action  by  investigating 
his  full  problem,  searching  out  objectives  and  alternatives,  and  comparing  them  in  the  light  of 
their  consequences,  using  an  appropriate  framework  ...to  bring  expert  judgment  and  intuition 
to  bear  on  the  problem.  This  suggests  that  a  cycle  consisting  of  definition  of  objectives, 
exploration  and  evaluation  of  alternatives  in  terms  of  their  costs  and  effectiveness  needs  to  be 
considered.  The  cycle  may  be  repeated  in  the  light  of  new  information  required  while  redefining 
the  objectives  and  identification  and  evaluation  of  alternatives,  till  the  total  spectrum  is 
completely  understood ,"9 

This  is  an  application  to  the  decision-making  process  of  the  scientific  method  used  by 
scientists  to  investigate  natural  phenomena. 

23.  The  scientific  method10  starts  by  performing  an  analysis  of  the  system  under 
examination.  This  means  that  the  system  is  split  into  manageable  parts  that  can  be 
observed  and  investigated  independently.  These  observations  underpin  the  formulation  of 
a  model  that  explains  the  system's  behaviour.  This  model  is  then  tested  via  experiment  for 
agreement  with  the  facts,  and  adjusted  as  necessary.  This  cycle  continues,  in  an  attempt  to 
eventually  completely  explain  the  behaviour  of  the  system.  It  is  important  to  note  that  the 
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model  is  constantly  re-evaluated  against  observations  made,  and  if  found  lacking,  is 
modified.  A  model  should  be  viewed  as  a  dynamic  item  within  the  experimental  process. 

24.  An  aspect  of  the  culture  of  science  is  the  principle  of  openness.  "The  method  and 
tradition  of  science  require  that  results  be  openly  arrived  at,  so  that  any  other  scientist  can 
retrace  the  same  steps,  and  get  the  same  results".11  This  openness  allows  science  to  be  self- 
regulating,  in  the  sense  that  all  scientific  work  should  be  peer  reviewed,  so  that 
assumptions  and  methodology  can  be  discussed.  The  open  publication  of  scientific  work 
aids  in  achieving  the  goal  of  objectivity,  where  questions  of  the  validity  of  results  can  be 
raised  by  anyone,  regardless  of  their  relative  position  or  standing.  This  leads  to  the 
creation  of  an  atmosphere  where  robust  debates  can  be  had  about  the  solution  to 
problems.  The  nature  of  scientific  debate  is  such  that  potential  solutions  to  problems  are 
discussed  objectively  and  without  prejudice,  meaning  for  example  that  novel,  innovative 
and  "out  of  the  box"  solutions  can  be  debated,  and  not  dismissed  out  of  hand. 

25.  How  can  this  analytical  cycle  and  culture  be  applied  to  assist  decision-makers  in  long- 
range  Defence  planning?  Firstly,  the  reason  or  reasons  for  a  decision  needing  to  be  made 
are  analysed.  For  example,  a  decision  of  whether  or  not  to  adopt  the  concept  of  Network 
Enabled  Operations  (NEO)  would  be  driven  by  the  possibility  that  it  is  a  better  way  of 
performing  combat  than  the  currently  used  concepts.  This  statement  of  a  decision's 
"trigger"  puts  the  decision  into  context,  and  helps  direct  the  subsequent  analysis.  In  the 
NEO  example,  the  analysis  would  be  directed  towards  defining  exactly  what  "better" 
might  mean,  and  how  it  can  be  measured. 

26.  Furthermore,  decisions  made  in  long-range  planning  will  by  definition  have  long-term 
effects,  and  will  subsequently  have  consequences  for  other  parts  of  an  organisation. 
Therefore  the  decision  needs  to  be  examined  within  the  system  of  which  it  is  a  part.  This 
brings  to  light  all  of  the  factors  that  need  to  be  analysed,  and  brought  to  the  attention  of 
the  decision-maker.  As  OR  endeavours  to  investigate  systems  and  understand  the  way 
they  operate,  the  scientific  process  does  add  great  value  and  clarity  at  this  stage. 

27.  Once  the  nature  and  scope  of  the  decision  have  been  clearly  defined,  the  key 
arguments  relevant  to  the  decision  can  be  developed.  This  allows  the  step  of  performing 
research  and  collecting  data  to  begin.  The  value  that  OR  can  bring  at  this  point  is  the 
rigorous  and  unbiased  fashion  in  which  data  can  be  collected.  The  scientifically  minded 
investigator  should  be  able  to  find  and  evaluate  data  independently  of  current  wisdom. 
Depending  on  the  decision  to  be  supported,  these  data  may  range  from  performance 
specifications  at  the  "hard"  OR  end  of  the  spectrum,  all  the  way  through  to  collecting 
expert  opinion  at  the  "soft"  OR  end.  "Hard"  and  "soft"  OR  refer  to  quantitative  and 
qualitative  OR,  respectively.  The  nature  of  the  decision  will  also  determine  the  granularity 
of  data  required. 

28.  After  the  data  have  been  collected  and  analysed,  hypotheses  regarding  the  factors  that 
will  affect  the  decision  can  begin  to  be  formed.  This  stage  is  intellectually  equivalent  to  the 
forming  of  a  model  in  the  cycle  of  scientific  investigation.  In  some  cases  it  is  appropriate  to 
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build  hard  mathematical  models  of  the  system,  and  use  the  model  to  investigate  the  effects 
of  potential  changes.  This  generally  works  quite  well  for  operational  questions,  such  as  the 
best  way  to  organise  logistics.  The  questions  that  arise  in  higher-level  strategic  decisions 
may  not  be  so  amenable  to  mathematical  modelling.  Questions  at  this  level  typically 
contain  "soft"  issues,  such  as  a  high  level  of  behavioural  content  for  which  there  is  little 
theory.  There  also  may  simply  not  be  enough  facts  about  the  problem  to  produce  a  good 
mathematical  model.  Both  of  these  situations  could  apply  to  die  problems  considered  by 
long-range  planners. 

29.  OR  deals  with  this  situation  by  using  tools  that  are  appropriate  to  the  task.  The 
approach  is  still  to  formulate  a  representation  or  model  of  the  question  under 
consideration  but  this  model  may  be  expressed  in  the  form  of  an  influence  diagram,  rather 
than  mathematics.  What  is  common  to  both  the  "hard"  mathematical  and  the  "soft" 
models  is  that,  whatever  tools  and  techniques  are  used  to  build  and  articulate  them,  they 
will  be  developed  following  a  scientifically  sound  procedure. 

30.  Models  developed  in  a  scientifically  sound  fashion  are  built  upon  a  set  of  hypotheses. 
Hypotheses  must  be  formulated  in  such  a  way  that  they  are  testable  by  experiment.  It  is 
important  to  note  that  no  hypothesis  can  be  proven  true,  but  they  can  be  proven  false.  In 
practice,  this  means  that  the  results  of  an  experiment  can  provide  support  for  a  hypothesis 
but  not  proof  of  its  correctness.  As  soon  as  experimental  evidence  is  found  that  contradicts 
the  hypothesis,  a  new  hypothesis  must  be  formulated  that  is  consistent  with  the  new 
results  whilst  still  being  consistent  with  all  previous  observations, 

31.  The  way  an  experiment  is  tun  should  be  determined  by  the  hypothesis  that  needs 
testing,  and  the  level  of  detail  that  is  required.  Hypotheses  flow  from  investigation  into  a 
particular  question,  and  the  level  of  detail  required  will  depend  on  the  decision  to  be 
supported.  Therefore,  it  is  important  that  any  experimentation  activity  have  a  clear 
purpose,  in  the  form  of  a  decision  that  it  will  support  of  knowledge  that  it  will  seek  to 
discover.  It  is  possible  to  engage  in  undirected  activities  in  the  hope  that  some  interesting 
information  will  appear,  but  this  is  usually  an  inefficient  and  wasteful  use  of  resources 
that  has  a  low  chance  of  success.  Having  a  decision  to  inform  also  means  that  relevant 
metrics  or  measures  of  performance  can  be  proposed,  and  it  will  be  clear  where  the 
information  gained  from  the  experiment  is  to  be  used. 

32.  Experimental  design  should  be  done  in  a  fashion  that  allows  the  actual  hypothesis  to 
be  tested  independently  of  other  factors.  For  example,  a  seminar  wargame  to  test  a  new 
concept  introduces  extra  variables,  such  as  the  concrete  force  structure  to  be  used  for  the 
purposes  of  the  game,  and  the  judgments  of  the  players.  The  experimental  procedure 
should  ideally  include  a  method  for  factoring  out  these  variables,  so  that  only  the  value  of 
the  concept  is  tested.  Doing  this  may  require  the  application  of  several  experimental 
techniques,  rather  than  being  limited  to  one.  Because  a  good  scientific  methodology  keeps 
track  of  any  extra  variables  introduced  into  an  experiment,  it  is  valuable  to  have  an  OR 
analyst  involved  in  the  experimental  design. 


5 


DSTO-CR-0282 


33.  Producing  a  good  and  robust  experimental  procedure  does  not  happen  by  magic.  Prior 
to  an  experiment  being  performed,  a  quantity  of  analysis  must  be  done  to  guide  and 
inform  the  subsequent  experimentation.  This  analysis,  which  might  be  referred  to  as  pre- 
experimental  analysis,  should  concern  itself  with  several  issues.  Firstly,  it  should  take  a 
system  perspective,  articulating  all  of  the  factors  that  will  influence  the  decision.  This 
provides  a  clear  statement  of  what  the  research  and  experimentation  should  set  out  to 
achieve,  and  provides  the  link  between  experimentation  and  the  decision.  Secondly,  a 
hypothesis  needs  to  be  expressed  in  a  form  that  can  be  tested.  Formulating  a  hypothesis 
requires  analysis  of  the  information  understood  prior  to  an  experiment  being  run.  This 
hypothesis  may  contain  many  assumptions  and  unknowns,  but  without  a  hypothesis,  no 
meaningful  experiments  can  be  performed.  The  final  piece  of  analysis  involves  the 
definition  of  metrics  or  measures  of  performance.  These  should  be  related  to  die 
hypothesis  under  consideration  in  an  experiment,  and  the  drivers  of  the  decision. 
Performing  this  pre-experimental  analysis  is  vital  to  informing  experimental  design. 

34.  At  the  end  of  a  set  of  experiments,  the  post-experimental  analysis  should  lead  to  the 
hypotheses  under  consideration  being  adjusted  to  account  for  the  observations  that  arise, 
and  to  inform  subsequent  experimentation.  The  cycle  of  analysis,  experimentation  and 
revising  hypotheses  can,  in  principle,  continue  indefinitely.  However,  at  some  point  the 
conclusions  of  the  research  and  experimentation  will  be  presented.  Due  to  time  and 
resource  constraints,  all  of  the  hypotheses  made  might  not  have  been  tested.  In  addition, 
there  will  always  be  factors  and  variables  that  cannot  be  measured  completely.  Thus  any 
results  will  contain  some  level  of  uncertainty,  whether  that  be  in  the  form  of  assumptions, 
expert  opinions,  or  the  like.  A  well  structured  OR  study  will  keep  track  of  these 
uncertainties  and  present  them  along  with  the  conclusions,  by  providing  some  comment 
on  the  level  of  confidence  of  the  results.  A  good  study  will  also  perform  a  sensitivity 
analysis,  which  asks  the  question:  how  much  do  my  conclusions  change  if  some  variables 
in  the  model  were  to  change? 

35.  Including  this  last  step  is  vital  as  it  expresses  the  level  of  risk  in  a  given  decision.  For  a 
decision-maker,  a  part  of  risk  involved  in  pursuing  a  given  option  is  related  to  what  is 
unknown.  The  conclusions  of  an  OR  study  should  include  the  pros  and  cons,  as  well  as  the 
level  of  risk  associated  with  each.  These  factors  can  then  be  evaluated  by  the  judgment  of 
the  decision-maker.  Note  that  OR  does  not  remove  the  need  for  judgment  in  a  decision, 
but  it  should  provide  the  decision-maker  with  a  more  robust  appreciation  of  the  drivers  of 
the  decision,  arid  a  greater  understanding  of  the  advantages  and  risks  associated  with  a 
given  course  of  action.  This  understanding  allows  the  decision-makers  to  have  a  robust 
debate  regarding  the  options  based  on  as  much  fact  as  possible,  rather  than  relying  solely 
on  judgments. 
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4.  Organisational  Implications  of  Operations  Research 


36.  Decision-making  Culture.  The  desirability  of  a  structured  and  rigorous  approach  to 
decision-making  is  self-evident  to  most  OR  practitioners  and  strong  arguments  from  a 
practitioner's  perspective  can  be  mounted  to  support  this  position.  However,  it  is  by  no 
means  clear  that  senior  decision-makers  share  this  point  of  view.  The  recently-retired  Vice 
Chief  of  the  Defence  Force,  LTGEN  Des  Mueller,  has  expressed  this  in  blunt  terms: 

"You  should  therefore  remind  yourselves  that  decisions  on  major  matters ...  will  he  made  at 
levels  where  intuitive  judgement  and  not  detailed  analysis,  with  the  possible  exception  of 
financial  analysis,  tends  to  prevail.  The  reality  of  this  has  precipitated  deep  despair  in  the 
heart  of  many  an  operations  research  analyst."12 

37.  While  this  may  be  starkly  expressed  for  effect,  the  history  of  quantitative  OR  support 
provided  directly  to  senior  decision-makers  through  the  committee  process1has  not  been 
encouraging  for  the  OR  community,  and  lends  some  weight  to  LTGEN  Mueller's 
sentiment.  Figure  1  illustrates  the  decline  in  the  number  of  quantitative  analysts  providing 
such  support  to  central  Defence  decision-makers  that  has  occurred  over  the  last  twenty- 
odd  years.  Whether  the  decline  has  occurred  because  the  quality  or  focus  of  the  analyses 
were  not  appropriate,  or  that  the  results  were  at  odds  with  political  imperatives,  or  the 
decision-making  environment  was  not  suited  to  the  use  of  analysis  (or  any  number  of 
other  reasons)  is  not  clear.  This  decline  does  seem  to  indicate  that,  over  time,  the  decision¬ 
making  culture  of  Defence  has  moved  away  from  using  analysis.  OR  is  more  likely  to  have 
a  positive  impact  if  this  trend  can  be  reversed. 

38.  OR  could  be  incorporated  into  an  organisation's  decision-making  processes  in  a 
number  of  ways  and  with  a  variety  of  expectations.  Each  of  these  have  implications  for  the 
organisation  (and  for  the  OR  analysts)  that  need  to  be  understood  and  considered  before 
implementing  any  particular  OR  regime.  If  the  implementation  is  done  well,  it  could  help 
reverse  the  trend  mentioned  above. 

39.  What  is  independent  analysis?13  Because  it  is  an  application  of  the  scientific  method, 
OR  should  provide  support  to  decision-makers  that  is  objective  and  impartial.  However,  the 
related  issue  of  independent  analysis  can  have  a  significant  impact  on  the  ability  of  OR  to 
provide  useful  support  to  decision-makers.  There  are  several  dimensions  to  independent 

1  The  data  in  Figure  1  refer  to  the  number  of  analysts  whose  work  was  represented  directly  at  committee  by  a 
two-star  position.  This  was  initially  Controller  Military  Studies  and  Operational  Analysis,  then  Controller, 
Projects  and  Analytical  Studies,  then  Controller,  External  Relations,  Projects  and  Analytical  Studies,  then  FAS 
Force  Development  and  Analysis,  then  Head  Capability  Plans  and  Resource  Programming,  then  Head, 
Strategic  Policy  and  Plans  (in  actuality,  DEPSEC  S)  and  finally  FAS  Capability  Investment  and  Resources. 
Governance  arrangements  for  the  input  of  analytical  advice  from  DSTO  to  the  committee  process  are 
fundamentally  different,  as  the  accountability  goes  via  the  sponsor's  desk  officer.  This  is  related  to  the  gradual 
downgrading  of  DSTO's  position  on  the  relevant  committee  from  "accountable  for  various  inputs"  to 
"advisory"  only,  and  its  general  thrust  as  a  support  organisation. 
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analysis,  each  of  which  can  lead  to  dysfunctional  conditions  for  providing  sound  advice 
appropriate  for  the  Defence  organisation  as  a  whole  if  they  are  not  handled  appropriately. 


1965  1970  1975  1980  1985  1990  1995  2000  2005 


Year 

Figure  1:  Number  of  analysts  providing  quantitative  analytical  support  directly  to  central  Defence 
decision-makers  through  the  committee  process  as  a  function  of  time.14  The 
organisational  units  that  were  responsible  for  providing  quantitative  analysis  are 
indicated  at  the  top  of  the  graph.  1974-75  marks  the  period  of  the  Tange  reviews  and  the 
formation  of  the  Department  of  Defence.  Data  for  the  years  prior  to  1974  refer  to 
analysts  in  the  Department  of  Supply. 

40.  Firstly,  if  the  decision-makers  consider  the  advice  they  get  as  irrelevant  and  only  good 
for  "cosmetic"  purposes,  the  value  of  the  advice  is  bound  to  be  minimal.  Under  such 
circumstances  the  OR  analysts  could  be  said  to  be  too  independent,  in  that  the  decision¬ 
makers  of  the  organisation  do  not  see  the  advice  to  be  of  relevance  to  them.  In  this  case  the 
independence  has  become  mutual  and  absolute,  and  is  definitely  counterproductive  to 
generating  favourable  conditions  for  OR-based  advice. 

41.  Secondly,  generation  of  good  advice  requires  free  access  to  all  relevant  input  data  and 
critical  assumptions,  an  ability  to  make  sound  judgements  on  their  validity  and 
appropriateness,  and  the  freedom  to  exercise  the  analyst's  best  professional  knowledge 
and  judgement.  This  aspect  of  analytical  independence  implies  an  ability  to  formulate  and 
assess  the  goals  of  the  organisation,  expressed  in  terms  suitable  for  analytical  purposes. 
Parts  of  an  organisation  that  perceive  that  the  results  of  an  analytical  approach  may 
threaten  their  comfortable  status  quo  may  decline  to  provide  information  (or  even  provide 
misinformation)  for  analysis,  and  thereby  undermine  the  validity  or  usefulness  of  the 
analysis.  Another  potential  barrier  to  objectivity  is  that  the  requirements  of  a  military 
command  structure  may  hinder  the  open  debating  of  hypotheses  and  results,  with  the 
possible  result  that  potentially  fruitful  solutions  might  never  be  considered. 
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42.  Thirdly,  perhaps  the  most  important  factor  to  be  considered  is  that  of  resource 
independence.  In  every  organisation  resources  are  limited  and  priorities  must  be  made 
both  between  OR  and  all  other  activities  in  the  organisation,  and  within  the  broad  range  of 
possible  OR  projects.  The  dysfunctional  conditions  that  are  of  greatest  concern  here  are 
those  that  can  arise  when  the  priorities  for  OR  work  are  affected  by  the  actual  results  of 
analyses  and  the  content  of  advice  given  as  a  result.  The  most  obvious  example  would  be 
for  the  organisation  to  link  future  priorities  of  work  to  getting  the  "desired"  results  from 
present  analyses,  or  to  put  it  bluntly:  buying  the  analysis  needed  to  support  a  priori 
conclusions. 

43.  If  OR  is  to  support  a  planning  framework  it  must  be  able  to  survive  within  the  political 
environment  of  which  it  is  a  part.  The  nature  of  objective  analysis  means  that  at  some 
point  OR  activities  will  undoubtedly  create  tension  with  other  areas  of  Defence  (it  is  not 
possible  to  please  all  of  the  people  all  of  the  time).  In  a  large  bureaucratic  organisation 
such  as  Defence,  this  is  guaranteed  to  generate  political  attacks  on  the  OR  group.  If  OR 
and  analysis  are  to  be  incorporated  as  an  essential  part  of  a  long-range  planning 
framework  there  must  be  strategies  in  place  to  deal  with  these  pressures. 

44.  What  is  the  value  of  independent  analysis?13  If  an  organisation  is  to  develop  and 
improve  to  remain  relevant  and  competitive  as  the  world  around  it  changes,  it  must  have 
mechanisms  in  place  that  will  allow  it  to  innovate.  It  is  vital  that  conventional  wisdom  is 
challenged  and  new  concepts  and  radical  solutions  are  raised  and  thoroughly  discussed. 
Good  OR  can  make  significant  contributions  to  these  forces  of  potentially  beneficial 
change  within  an  organisation,  by  providing  the  forum  for  a  robust,  scientific  debate  of 
questions  and  their  solutions.  Enabling  this  debate  would  be  useful  for  long-range 
planning,  as  it  helps  in  the  process  of  achieving  cultural  change  by  providing  the  means  to 
escape  the  limits  of  accepted  wisdom.  In  doing  so,  however,  the  advice  provided  by 
analysts  is  very  often  not  wanted  by  those  parts  of  the  organisation  that  might  be 
candidates  for  change:  change  can  be  difficult  and  uncomfortable.  If  the  OR  group  is 
dependent  on  those  elements  of  the  organisation  for  the  conduct  of  its  analyses,  or  even  for 
its  continued  existence,  unwanted  but  relevant  advice  may  be  a  rare  deliverable. 

45.  The  utility  for  the  organisation  as  a  whole  of  OR  contributions  to  planning  and 
decision-making  hinges  on  arrangements  that  secure  the  production  and  delivery  of  all 
relevant  results  and  advice,  irrespective  of  the  degree  to  which  it  is  wanted  by  the 
organisational  elements  in  question.  This  implies  that  the  organisational  culture  is  such 
that  open  and  robust  debate  and  questioning  is  not  only  acceptable  but  encouraged.  It  can 
pose  a  significant  challenge  to  management  to  encourage  such  a  culture  to  flourish  if  it  is 
not  already  doing  so,  but  without  it  the  value  of  OR  is  unlikely  to  be  realised.  It  should  be 
noted  that  the  propensity  to  repel  certain  types  of  advice  is  by  no  means  a  "constant  of 
nature"  in  organisations.  Indeed,  it  varies  from  one  organisation  to  another,  and  also 
varies  over  time  as  a  function  of  the  internal  "climate"  of  the  organisation. 

46.  Independent  scrutiny  vs  doing  the  analysis.  Analysts  can  provide  an  independent 
review  function  within  an  organisation,  essentially  providing  an  assessment  of  the  rigour 
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of  a  process  and  the  conclusions  that  are  drawn  from  it,  or  they  can  be  an  integral  part  of 
developing  processes  and  conducting  analyses.  If  analysts  have  provided  advice,  assisted 
in  the  development  of  processes  and/or  conducted  analyses,  it  is  not  possible  for  them 
then  to  provide  an  assessment  or  review  of  the  rigour  of  the  work  without  a  real  or 
apparent  conflict  of  interest.  The  organisation  for  which  the  analysts  work  must  be  very 
clear  about  what  role  the  analysts  are  expected  to  play:  "doers"  or  "reviewers",  but  not 
both.  If  both  roles  are  required,  the  organisation  should  identify  two  distinct  groups  of 
analysts  to  perform  the  two  functions. 

47.  Impartial  analysts  or  advocates?15  Ethical  dilemmas  can  arise  as  a  result  of  the  manner 
in  which  OR  analysts  present  the  results  of  their  work.  Three  general  cases  can  be 
envisaged: 

•  the  analyst  presents  the  facts  in  an  objective  and  unbiased  manner  and  do  not 
advocate  any  particular  position; 

•  the  analyst  has  a  moral  responsibility  for  die  effects  of  his  own  involvement;  and 

•  the  analyst  works  openly  as  an  advocate  for  a  particular  position. 

48.  As  scientists,  OR  analysts  usually  find  themselves  in  the  first  situation,  presenting 
objective  analyses  and  leaving  it  to  the  decision-maker  to  make  their  judgements.  When 
working  in  policy  areas,  however,  analysts  often  find  themselves  operating  in  the  second 
of  these  regimes  and  because  of  the  nature  of  the  work  must  take  great  care  that  personal 
biases  are  not  embedded  or  implied  in  die  analysis.  When  the  analyst  works  explicitly  as 
an  advocate,  for  example  as  an  expert  witness  or  adversary,  he  is  guided  principally  by  his 
personal  code  of  ethics.  There  is  no  imperative  to  volunteer  all  the  relevant  information  in 
this  situation,  only  that  which  supports  the  position  being  advocated .  The  analyst  needs  to 
know  whether  he  is  employed  as  an  impartial  provider  of  analysis,  or  an  advocate,  or 
something  in  between.  It  is  also  important  for  both  the  analyst  and  the  organisation  for 
which  he  works  to  understand  the  potential  pitfalls  that  may  arise  when  the  analyst  is 
operating  at  any  given  point  on  this  continuum. 

49.  Tool  development  vs  analysis.  Anecdotal  evidence16  suggests  that  there  are 
approximately  200  OR  practitioners  currently  employed  by  Defence,  principally  in  DSTO. 
The  majority  of  these  are  tool  development  oriented,  providing  support  to  projects,  tactics 
development  or  training  rather  than  engaged  in  providing  analytical  support  for  decision¬ 
makers.  The  question  arises  as  to  where  to  place  the  balance  of  the  OR  effort  for  long-term 
planning,  into  tool  development  for  long-range  planning,  or  into  actually  doing  the 
analysis. 

50.  Quantitative  vs  qualitative  analysis.  In  the  preceding  discussion  it  has  been  implicitly 
assumed  that  "hard"  and  "soft"  (quantitative  and  qualitative,  respectively)  OR  are  of 
essentially  equal  value  to  the  decision-making  process.  Whilst  the  characteristics  of  a 
structured  scientific  approach  to  both  quantitative  and  qualitative  analysis  might  be 
similar,  the  problems  to  which  they  are  applied,  the  tools  that  might  be  used  and  the 
results  that  can  be  expected  are  not.  Quantitative  OR  is  more  applicable  in  situations 
where  models  are  predictive  to  a  reasonable  extent,  while  qualitative  OR  applies  where 
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models  are  more  exploratory  in  nature.  Quantitative  OR  often  results  in  numerical  data 
that  are  useful  for  underpinning  decisions  about  relatively  concrete  issues  (the  results  of 
the  EO  Stockholding  study  are  an  example),  while  qualitative  OR  can  provide  clarity  and 
insights  into  a  problem  but  rarely  provides  numerical  support  for  decisions  (analysis  of  a 
future  warfighting  concept  might  be  an  example).  The  nature  of  the  decisions  to  be 
supported  will  to  some  extent  dictate  which  end  of  the  analytical  spectrum  is  appropriate. 

51.  Analysis  should  be  conducted  with  rigour  regardless  of  whether  it  is  at  the  qualitative 
or  quantitative  end  of  the  spectrum.  However,  rigour  should  not  be  confused  with 
accuracy  or  precision.  Accuracy  relates  to  the  degree  to  which  results  accord  with  reality, 
while  precision  refers  to  the  definiteness  with  which  an  answer  is  given.  An  example  of  a 
precise  and  accurate  model  is  one  that  makes  numerical  predictions  that  are  in  very  close 
agreement  with  subsequent  observations.  It  is  entirely  possible,  however,  to  achieve 
accuracy  without  precision,  and  precision  without  accuracy.  Qualitative  analysis  will 
almost  certainly  be  less  precise  and  less  accurate  than  quantitative  analysis.  The  critical 
point  is  that  the  nature  of  the  analysis  conducted  to  support  a  particular  decision  should, 
as  far  as  possible,  match  the  decision-maker's  needs  for  accuracy  and  precision.  Some 
decisions,  for  example,  can  be  informed  using  qualitative  analysis  and  don't  require  high 
degrees  of  certainty  (eg.  futures  analysis)  but  the  analysis  should  still  be  conducted  with 
rigour. 

52.  The  reality  is  that  all  forms  of  analysis  will  have  some  level  of  imprecision  and 
inaccuracy  involved,  if  for  no  other  fact  than  models  are  just  representations  of  the  real 
world  rather  than  reality  itself.  The  way  that  this  can  be  dealt  with  is  by  expressing  some 
measure  of  the  confidence  of  the  conclusions  or  the  outputs.  In  quantitative  analysis,  this 
can  be  done  relatively  easily,  in  the  form  of  error  bars  or  confidence  intervals.  For 
qualitative  analysis  this  is  somewhat  more  difficult.  Qualitative  methods  such  as 
wargaming  represent  complex  systems  usually  involving  human  interaction,  and  running 
through  a  scenario  just  once  is  unlikely  to  provide  an  accurate  answer.  The  way  that 
qualitative  and  quantitative  analysis  can  deal  with  uncertainty  should  have  some 
influence  over  which  types  of  analysis  are  used. 

53.  What  decisions  are  to  be  supported?  Senior  committees  and  decision-makers  have  a 
legal  obligation  to  make  year  zero  resource  decisions  and  to  provide  assurance  that  further 
year  funding  will  be  adequate  to  support  the  results  of  those  decisions.  In  making  these 
decisions,  senior  executives  are  responsible  for  ensuring  that  they  are  consistent  with 
Government  guidance  and  for  presenting  a  range  of  options  and  recommending  or 
proposing  the  most  cost  effective  option  in  net  present  value  terms  for  Government 
consideration.  They  are  also  not  to  mislead  Government  when  providing  assurance  of 
further  year  binding  to  operate  whatever  is  implemented.  However,  the  in-year  resource 
decisions  appear  to  be  accorded  a  higher  priority  and  importance  than  the  assurance  of 
future  affordability,  possibly  because  subsequent  decisions  can  address  problems  caused 
by  previous  assurances  of  affordability  that  turned  out  to  be  inaccurate. 
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54.  Because  long-range  planning  has  a  time  horizon  significantly  beyond  the  immediate 
year  zero  resource  decisions  that  the  senior  executives  are  focussed  on,  it  can  be  difficult  to 
engage  them  in  such  planning  and  for  them  to  see  immediate  value  in  it.  It  is  also  possible 
that  the  analytical  community  has  done  a  poor  job  of  articulating  the  implications  of  long¬ 
term  issues  for  decision-makers  or  of  breaking  down  the  uncertainty  by  exploring  the 
future  space  systematically. 

55.  OR  has  the  potential  to  add  value  to  decision  support  when  resource  constraints  are 
explicitly  addressed  in  the  analysis.  It  could  do  this  through  application  of  realistic 
resource  constraints  in  the  initial  development  of  options  or  through  some  form  of  cost- 
effectiveness  analysis  of  options,  regardless  of  how  those  options  were  developed. 

56.  How  much  OR  is  enough?  For  OR  to  have  relevance  to  decision-makers  it  must 
deliver  results  that  could  make  a  material  difference  to  the  decision  in  time  to  inform  the 
decision.  The  academic  110%  solution  delivered  3  years  after  the  decision  is  made  is  of  no 
use,  but  an  80%  solution  delivered  in  reasonable  time  may  offer  useful  decision  support. 
This  issue  of  delivering  timely  advice  is  crucial,  because  of  the  nature  of  Defence's 
decision-making  process,  which  is  influenced  by  the  rapidly  changing  policy  environment. 

57.  All  decisions  will  involve  both  judgement  and  analysis.  For  senior  executives  the 
balance  lies  towards  judgement,  while  for  analysts  the  balance  lies  towards  analysis.  For 
senior  executives  to  place  more  emphasis  on  the  results  of  analysis  in  making  their 
decisions  the  analysis  will  have  to  demonstrate  its  value.  Perhaps  the  principal  way  it  can 
do  that  is  to  provide  them  with  objective  and  transparent  decision  support  material  to 
underpin  and  augment  their  judgements.  This  would  provide  a  more  readily  auditable 
basis  for  decisions  that  could  more  readily  be  communicated  with  Government,  which 
currently  has  little  confidence  in  many  of  the  decisions  taken  by  Defence. 

58.  Resources.  There  are  never  enough  resources  to  do  everything.  The  cyclic  nature  of  the 
scientific  method  means  that  not  insignificant  resources  might  be  required  to  do  OR  well. 
Realistic  choices  must  be  made  regarding  the  scope  and  scale  of  activities  to  be 
undertaken.  The  question  to  be  answered  here  is  where  the  best  value  can  be  obtained  for 
die  limited  resources  available  for  analysis. 


5.  Integrating  Analysis  into  Concept-Led  Long-Range 

Planning 

59.  The  cultural  and  ownership  issues  discussed  previously  need  to  be  dealt  with  if 
analysis  is  to  be  integrated  into  an  organisation.  The  solutions  to  these  problems  will 
depend  to  a  large  extent  on  how  analysis  is  incorporated  into  the  planning  framework  (if 
at  all)  and  what  the  organisation  expects  to  gain  from  analysis.  What  we  wish  to  present  is 
a  possible  mindset  regarding  analysis  and  experimentation  that  will  help  guide  its  position 
in  a  planning  framework.  We  also  argue  that  this  mindset  will  take  steps  to  dealing  with 
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some  of  the  above  cultural  problems.  The  key  element  to  this  mindset  is  that  the  analysis 
should  take  a  decision  perspective  (see  Figure  2). 


Figure  2:  A  structured  approach  to  analysis  in  support  of  decision-making.17 


60.  The  idea  behind  taking  a  decision  perspective  is  that  research  and  analysis  should  be 
directed  towards  possible  decisions,  rather  than  searching  for  knowledge  generally.  It 
should  guide  the  analysis  to  provide  the  understanding  that  decision-makers  need  in 
terms  of  both  breadth  and  depth,  and  explicitly  discuss  risk.  Note  that  the  framework 
indicates  possible  rather  than  actual  decisions.  The  reason  for  this  is  that  the  lead-time  to 
provide  appropria  te  material  for  decision  support  may  be  lengthy,  so  analysis  may  need  to 
pre-empt  decisions.  This  approach  will  also  support  analysis  for  long-range  planning, 
where  the  issues  considered  may  have  little  immediate  committee  visibility.  What  an 
analytical  procedure  should  do  is  imagine  a  possible  decision  that  might  arise,  and  refine 
its  methodology  to  provide  information  relevant  to  the  key  arguments,  that  is,  the  factors 
affecting  a  decision. 

61.  The  way  this  canbe  achieved  is  to  make  hypotheses  regarding  the  key  arguments,  and 
use  research  and  experimentation  to  test  these  hypotheses.  For  example,  a  decision  on 
whether  to  adopt  a  new  warfighting  concept  would  be  based  on  its  relative  effectiveness. 
The  hypothesis  to  be  tested  would  be  that  the  new  concept  is  more  effective  than  those 
currently  in  use.  A  program  of  experimentation  and  analysis  could  then  be  developed  to 
test  this  hypothesis.  Because  the  hypothesis  has  been  linked  to  a  possible  decision,  the 
analysis  and  experimentation  done  should  be  useful  to  long-range  planning.  Allowing  the 
culture  of  science  to  work  here  should  also  encourage  robust  debate  to  take  place,  which 
helps  the  development  of  new  ideas,  and  hence  good  solutions  to  problems. 
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62.  Because  decision-makers  should  consider  the  implications  of  a  decision  for  the  entire 
organisation,  analysis  and  experimentation  linked  to  a  decision  should  also  take  a  system 
perspective.  This  simply  means  that  analysis  should  take  into  account  the  whole  system, 
from  the  strategic  level,  through  to  capability  considerations  if  appropriate.  There  should 
also  be  considerations  of  the  long-term  effects  of  current  decisions.  Clearly  the  way  a 
particular  piece  of  analysis  considers  the  system  will  depend  on  the  decision.  However, 
taking  a  system  perspective  has  implications  for  the  tools  and  techniques  used.  Ideally,  a 
family  of  appropriate  models,  simulations  and  analytic  methods  could  be  developed  and 
integrated,  so  that  knowledge  at  different  levels  and  from  different  perspectives  could  be 
readily  used  to  give  an  overall  description  of  the  implications  of  a  decision.  This  would 
require  analytical  tools  to  break  a  system  down  into  manageable  parts  that  can  be  studied 
more  or  less  independently,  tools  to  do  exploratory  analysis  for  breadth,  tools  to  test  high 
resolution  hypotheses,  and  finally,  integrative  techniques  to  build  up  the  entire  picture. 

63.  One  analytic  method  that  could  be  used  to  break  down  problems  into  manageable 
pieces  is  to  use  the  idea  of  layered  hypotheses.  For  example,  the  NEO  concept  would 
involve  measuring  its  effectiveness.  The  NEO  concept  is  made  up  of  several  smaller 
building  blocks,  such  as  the  ability  to  conduct  secure  communications,  and  the  ability  to 
filter  information  appropriately  to  avoid  information  overload.  These  requirements  may  in 
turn  depend  on  other  lower  level  factors.  All  of  these  lower  levels  could  be  analysed,  and 
the  combination  of  these  different  levels  could  be  used  to  develop  an  overall  picture  of  the 
viability  of  the  concept.  Performing  these  different  levels  of  analysis  will  require  a 
significant  investment  of  resources,  in  terms  of  development  of  appropriate  analytical 
techniques,  and  the  performance  of  the  analysis  itself.  It  maybe  necessary  to  develop  tools 
on  an  ad  hoc  basis,  depending  on  the  decision  and  the  hypothesis,  rather  than  relying  on  a 
suite  of  predefined  methods. 

64.  An  example  of  the  way  taking  the  decision  perspective  drives  the  resource 
requirement  for  analysis  is  the  issue  of  robustness.  For  example,  a  positive  decision  on  a 
new  capability  is  unlikely  unless  it  can  be  demonstrated  to  be  flexible  and  robust.  Analytic 
activities  should  then  be  directed  towards  testing  a  capability  under  a  range  of  conditions. 
Experiments  could  be  based  on  hypothetical  situations,  in  order  to  stretch  options  and  see 
where  they  break.  For  example,  an  NEO  force  structure  may  be  hypothetically  placed  in 
situations  where  enemy  jamming  has  disrupted  friendly  communications,  or  a  wargame 
may  include  a  Red  Team  fighting  in  an  innovative  way.  Therefore,  there  may  need  to  be  a 
wide  range  of  experiments  for  a  single  decision,  adding  to  the  resource  requirements. 

65.  Finally,  this  approach  creates  the  need  for  analysis  to  be  embedded  in  the  long-range 
planning  process.  Adopting  the  decision  perspective  mindset  means  that  there  is  a 
requirement  for  analysis  to  be  done  at  each  of  the  above  stages,  so  that  the  factors  driving 
a  possible  decision  can  be  broken  down  to  the  point  where  experimentation  can  be 
usefully  applied. 

66.  A  consequence  of  adopting  the  decision  perspective  is  that  the  analysis  performed  will 
have  greater  chance  of  being  considered  useful,  as  it  will  be  better  directed,  ie.  focussed  on 
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providing  information  that  the  decision-maker  might  require.  It  may  also  assist  in  raising 
the  profile  of  strategic  issues  by  explicitly  linking  them  to  current  decisions. 

67.  The  structured  approach  to  analysis  illustrated  in  Figure  2  makes  no  assumptions 
about  the  nature  of  the  planning  process:  it  is  equally  applicable  to  a  variety  of  planning 
philosophies,  such  as  top-down,  bottom-up  or  resource-constrained.  What  will  be  different 
in  these  cases  is  the  nature  of  the  decision,  and  the  means  by  which  they  are  generated.  A 
concept-led  long-range  planning  process,  however,  will  be  an  explicitly  top  down 
approach,  and  if  it  is  to  be  integrated  into  the  organisation,  must  interface  with  the  bottom- 
up  and  resource-constrained  processes  that  already  exist,  notably  in  Capability  Systems. 

68.  This  approach  to  analysis  is,  of  course,  no  guarantee  of  success.  There  will  still  be 
resistances  to  OR  being  involved  in  Defence's  planning,  be  it  long-range  or  otherwise, 
some  of  which  have  been  outlined  in  the  previous  section.  Having  the  analysis  done  with  a 
decision  perspective  though  is  vital  to  giving  it  any  chance  of  survival. 


6.  Conclusions 

69.  This  paper  has  discussed  what  OR  aims  to  do:  that  is,  use  the  intellectual  rigour  of  the 
scientific  method  in  support  of  decision-making.  The  scientific  method  follows  a 
continuous  cycle,  using  analysis  to  form  hypotheses,  testing  hypotheses  in  experiments, 
and  analysing  experimental  results  to  form  new  hypotheses.  This  cycle  means  that 
analysis  should  guide  experimentation,  rather  than  the  current  practice  of  performing 
experimentation  not  linked  to  clear  questions.  Ideally,  experimentation  should  be 
incorporated  into  the  methodology  of  doing  analysis,  rather  than  existing  as  a  separate 
activity. 

70.  Analytical  work  done  to  aid  any  planning  framework  should  take  a  decision 
perspective.  Establishing  a  link  to  a  possible  decision  focuses  the  analysis  on  producing 
information  that  the  decision-makers  might  find  useful,  by  forcing  the  analytical 
methodology  to  adjust  itself  to  the  decision-maker's  perspective.  It  also  allows  analysis  to 
be  done  for  any  decision  that  needs  to  be  informed,  meaning  that  it  is  possible  to  embed 
analysis  in  the  planning  process.  We  would  argue  that  embedding  rigorous  analysis  and  a 
culture  of  open  debate  in  the  process  is  important,  as  it  creates  an  environment  in  which 
novel  and  innovative  solutions  to  problems  can  be  discussed  in  a  way  that  helps  lead  to 
organisational  improvements.  In  long-range  planning,  this  could  help  the  debate 
regarding  new  concepts.  Embedding  analysis  will  also  be  required  for  the  decision 
perspective  as  analysis  will  need  to  be  done  at  each  level  from  splitting  the  decision  into  its 
key  arguments,  through  to  actually  performing  experiments. 

71.  There  are  several  decisions  to  be  made  regarding  the  role  of  analysis  in  long-range 
planning,  including  among  other  things  the  independence  of  the  analysts.  Those  issues 
will  all  have  some  influence  on  the  value  that  OR  can  add  to  the  planning  process,  both  in 
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terms  of  quality  of  work,  and  value  to  the  decision  maker.  One  consideration  that  will 
need  to  be  dealt  with  if  analysis  is  to  be  included  in  tire  framework  at  all  is  that  of 
resources.  Producing  good  analysis  and  running  a  useful  program  of  experimentation 
requires  people  and  dollars,  and  for  long-range  planning,  a  commitment  over  a  long  time. 


72.  There  are  elements  of  organisational  culture  within  Defence  and  the  analytical 
community  that  currently  mitigate  against  getting  full  value  from  analysis.  It  is  clear  from 
Figure  1  and  the  comment  from  LTGEN  Mueller  that  senior  executives  currently  see  little 
point  in  using  the  results  of  OR  as  it  is  currently  practised  to  inform  their  deliberations.  It 
also  appears  that  long-range  decisions  receive  less  attention  than  immediate  resource 
decisions,  a  situation  that  has  implications  for  a  long-range  planning  framework  in  its 
entirety.  However,  OR  has  the  potential  to  contribute  significantly  if  incorporated 
appropriately  into  the  long-range  planning  process,  and  if  it  can  demonstrate  its  utility 
maybe  a  catalyst  for  changes  in  the  decision-making  culture. 
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